Image processing system, projector, computer-readable storage medium, and image processing method

ABSTRACT

A projector is configured of a difference image generation section that generates a difference image between an all-white first calibration image and a second calibration image which includes an image having features at the center and periphery thereof, a center reference position detection section that detects a reference position of a center block area in the difference image, a peripheral reference position detection section that detects a reference position of a peripheral block area, and a projection area information generation section that generates projection area information indicating a position of a projection area, based on those reference positions.

This is a Division of application Ser. No. 11/083,033, filed Mar. 18,2005, which in turn claims the benefit of Japanese Patent ApplicationNo. 2004-095936, filed on Mar. 29, 2004, and Japanese Patent ApplicationNo. 2004-095937, filed on Mar. 29, 2004. The disclosure of the priorapplications is hereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

The present invention relates to an image processing system, aprojector, a computer-readable storage medium, and an image processingmethod in which the detection of a projection area is based on sensinginformation.

A recent proposal for adjusting the position of a projection imageinvolves using a projector that is provided with a CCD camera to projectan image onto a projection target such as a screen, and using the CCDcamera to sense the projected projection image and detect thecoordinates of four corners of the projection-target area thatcorrespond to the projection target within the sensing area.

For example, Japanese Patent Application Laid-Open No. 5-30520 proposesa configuration in which the position at which an image is projected isadjusted by displaying an image within an image-displayable area andsensing the entire image-displayable area.

However, when the configuration is based on the premise that the imageis displayed within the image-displayable area, as in Japanese PatentApplication Laid-Open No. 5-30520, it will not be possible to adjust theprojection position of the image if part of the image is projectedoutside of the image-displayable area.

In particular, since projectors have become smaller in recent years,users are treating them as mobile projectors by carrying them around andsetting them up in other locations to project images. In such a case,restrictions in installation locations could result in part of theprojection image being displayed outside the screen. In such asituation, it is necessary to have a projector that can adjust theposition of the projection image as appropriate.

In addition, although this Japanese Patent Application Laid-Open No.5-30520 does disclose the process of detecting the positions of fourcorners of the displayable area of the screen by fetching luminanceinformation stored in a frame memory into a CPU for image processing, itdoes not disclose any specific details of that image processing.

When the positions of the four corners of the screen are detected fromsensing information during general-purpose image processing used in theart, it is necessary to subject the entire sensed image to filteringprocessing, which takes time for the image processing and also greatlyincreases the amount of calculations.

For that reason, calibration is time-consuming and thus the user has towait for a longer time.

SUMMARY

According to a first aspect of the present invention, there is providedan image processing system comprising:

an image projection section which projects a first calibration imagetowards a projection target;

a sensing section which senses the projected first calibration image ata low resolution which is less than or equal to a predeterminedresolution, to generate first sensing information, and also senses thefirst calibration image at a high resolution which is greater than orequal to the low resolution, to generate third sensing information; and

a projection-target area detection section which generatesprojection-target area information relating to a position of aprojection-target area corresponding to the projection target in asensing area of the sensing section, based on the first and thirdsensing information,

wherein the projection-target area detection section comprises:

an edge detection section which performs edge detection to generatefirst edge detection information, based on the first sensinginformation, and also performs edge detection to generate third edgedetection information, based on the third sensing information; and

a projection-target area information generation section which generatesprovisional detection information by provisionally detecting theprojection-target area based on the first edge detection information,and also generates the projection-target area information based on thethird edge detection information,

wherein the edge detection section generates the third edge detectioninformation by performing edge detection on a pixel group around aboundary line of the provisionally detected projection-target area,based on the provisional detection information.

According to a second aspect of the present invention, there is provideda projector having the above image processing system.

According to a third aspect of the present invention, there is provideda computer-readable storage medium having a computer-executable programembedded thereon, the program including computer executableinstructions, when executed by a computer, causing the computer to:

an image projection control section which causes an image projectionsection to project a first calibration image towards a projectiontarget;

a sensing control section which causes a sensing section to sense theprojected first calibration image at a low resolution which is less thanor equal to a predetermined resolution, to generate first sensinginformation, and also for sensing the first calibration image at a highresolution which is greater than or equal to the low resolution, togenerate third sensing information; and

a projection-target area detection section which generatesprojection-target area information relating to a position of aprojection-target area corresponding to the projection target in asensing area of the sensing section, based on the first and thirdsensing information,

wherein the projection-target area detection section comprises:

an edge detection section which performs edge detection to generatefirst edge detection information, based on the first sensinginformation, and also performs edge detection to generate third edgedetection information, based on the third sensing information; and

a projection-target area information generation section which generatesprovisional detection information by provisionally detecting theprojection-target area based on the first edge detection information,and also generates the projection-target area information based on thethird edge detection information,

wherein the edge detection section generates the third edge detectioninformation by performing edge detection on a pixel group around aboundary line of the provisionally detected projection-target area,based on the provisional detection information.

According to a fourth aspect of the present invention, there is providedan image processing method comprising:

projecting a first calibration image towards a projection target;

sensing the projected first calibration image at a low resolution whichis less than or equal to a predetermined resolution by using a sensingsection, to generate first sensing information;

sensing the first calibration image at a high resolution which isgreater than or equal to the low resolution by using the sensingsection, to generate third sensing information;

performing edge detection to generate first edge detection information,based on the first sensing information;

generating provisional detection information by provisionally detectinga projection-target area corresponding to the projection target in asensing area of the sensing section, based on the first edge detectioninformation;

generating third edge detection information by performing edge detectionon a pixel group around a boundary line of the provisionally detectedprojection-target area, based on the provisional detection information;and

detecting the projection-target area to generate projection-target areainformation relating to a position of the projection-target area, basedon the third edge detection information.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIG. 1 is a schematic view of an image projection state of a firstembodiment of the present invention.

FIG. 2 is a conceptual view of the positional relationship of a screenand a projection image in accordance with the first embodiment.

FIG. 3 is a functional block diagram of a projector in accordance withthe first embodiment.

FIG. 4 is a block diagram of hardware for the projector in accordancewith the first embodiment.

FIG. 5 is a flowchart of position detection processing for a projectionarea in accordance with the first embodiment.

FIG. 6A is a conceptual view of a first calibration image, and FIG. 6Bshows a conceptual view of a second calibration image.

FIG. 7 is a conceptual view of a search method in a first stage of adetection of center reference positions in accordance with the firstembodiment.

FIG. 8 is a conceptual view of a search method in a second stage of thedetection of the center reference positions in accordance with the firstembodiment.

FIG. 9 is a conceptual view of a search method in a first stage ofdetection of peripheral reference positions in accordance with the firstembodiment.

FIG. 10 is a conceptual view of a search method in a second stage of thedetection of the peripheral reference positions in accordance with thefirst embodiment.

FIG. 11 is a conceptual view of a first stage of setting approximatedstraight lines in accordance with the first embodiment.

FIG. 12 is a conceptual view of a second stage of setting approximatedstraight lines in accordance with the first embodiment.

FIG. 13 is a functional block diagram of a projector in accordance witha modification of the first embodiment.

FIG. 14 is a conceptual view of a search method in a first stage of adetection of peripheral reference positions in a modification of thefirst embodiment.

FIG. 15 is a conceptual view of a search method in a second stage of thedetection of the peripheral reference positions in the modification ofthe first embodiment.

FIG. 16 is a functional block diagram of a projector in accordance witha second embodiment.

FIG. 17 is a flowchart of position detection processing for aprojection-target area in accordance with the second embodiment.

FIG. 18 is a conceptual view of a search method in a first stage ofdetection of the projection-target area in accordance with the secondembodiment.

FIG. 19 is a conceptual view of a search method in a second stage of thedetection of the projection-target area in accordance with the secondembodiment.

FIG. 20 is a conceptual view of a search method in a third stage of thedetection of the projection-target area in accordance with the secondembodiment.

FIG. 21 is a conceptual view of evaluation processing of edge detectionpoints in accordance with the second embodiment.

FIG. 22 is a conceptual view of a search method in a first stage ofdetection of peripheral reference positions in accordance with amodification of the second embodiment.

FIG. 23 is a conceptual view of a search method in a second stage of thedetection of the peripheral reference positions in accordance with themodification of the second embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT

Embodiments of the present invention were devised in the light of theabove-described technical problems. The embodiments may provide an imageprocessing system, a projector, a computer-readable storage medium, andan image processing method that enable the appropriate generation ofpositional information for a projection area, even when part of aprojection image is displayed outside of a projection target.

Furthermore, embodiments of the present invention may also provide animage processing system, a projector, a computer-readable storagemedium, and an image processing method that enable the generation ofpositional information for the projection-target area in a shorter timeand also accurately.

According to one embodiment of the present invention, there is providedan image processing system comprising:

an image projection section which projects a first calibration imagetowards a projection target;

a sensing section which senses the projected first calibration image ata low resolution which is less than or equal to a predeterminedresolution, to generate first sensing information, and also senses thefirst calibration image at a high resolution which is greater than orequal to the low resolution, to generate third sensing information; and

a projection-target area detection section which generatesprojection-target area information relating to a position of aprojection-target area corresponding to the projection target in asensing area of the sensing section, based on the first and thirdsensing information,

wherein the projection-target area detection section comprises:

an edge detection section which performs edge detection to generatefirst edge detection information, based on the first sensinginformation, and also performs edge detection to generate third edgedetection information, based on the third sensing information; and

a projection-target area information generation section which generatesprovisional detection information by provisionally detecting theprojection-target area based on the first edge detection information,and also generates the projection-target area information based on thethird edge detection information,

wherein the edge detection section generates the third edge detectioninformation by performing edge detection on a pixel group around aboundary line of the provisionally detected projection-target area,based on the provisional detection information.

According to one embodiment of the present invention, there is provideda projector having the above image processing system.

According to one embodiment of the present invention, there is provideda computer-readable storage medium having a computer-executable programembedded thereon, the program including computer executableinstructions, when executed by a computer, causing the computer to:

an image projection control section which causes an image projectionsection to project a first calibration image towards a projectiontarget;

a sensing control section which causes a sensing section to sense theprojected first calibration image at a low resolution which is less thanor equal to a predetermined resolution, to generate first sensinginformation, and also for sensing the first calibration image at a highresolution which is greater than or equal to the low resolution, togenerate third sensing information; and

a projection-target area detection section which generatesprojection-target area information relating to a position of aprojection-target area corresponding to the projection target in asensing area of the sensing section, based on the first and thirdsensing information,

wherein the projection-target area detection section comprises:

an edge detection section which performs edge detection to generatefirst edge detection information, based on the first sensinginformation, and also performs edge detection to generate third edgedetection information, based on the third sensing information; and

a projection-target area information generation section which generatesprovisional detection information by provisionally detecting theprojection-target area based on the first edge detection information,and also generates the projection-target area information based on thethird edge detection information,

wherein the edge detection section generates the third edge detectioninformation by performing edge detection on a pixel group around aboundary line of the provisionally detected projection-target area,based on the provisional detection information.

According to one embodiment of the present invention, there is providedan image processing method comprising:

projecting a first calibration image towards a projection target;

sensing the projected first calibration image at a low resolution whichis less than or equal to a predetermined resolution by using a sensingsection, to generate first sensing information;

sensing the first calibration image at a high resolution which isgreater than or equal to the low resolution by using the sensingsection, to generate third sensing information;

performing edge detection to generate first edge detection information,based on the first sensing information;

generating provisional detection information by provisionally detectinga projection-target area corresponding to the projection target in asensing area of the sensing section, based on the first edge detectioninformation;

generating third edge detection information by performing edge detectionon a pixel group around a boundary line of the provisionally detectedprojection-target area, based on the provisional detection information;and

detecting the projection-target area to generate projection-target areainformation relating to a position of the projection-target area, basedon the third edge detection information.

With this embodiment, the image processing system and the like cangenerate positional information for the projection-target area in ashorter time and also more accurately, by provisionally detecting theprojection-target area based on low-resolution sensing information thendetecting the projection-target area based on high-resolution sensinginformation in the vicinity of a boundary line thereof.

In the image processing system, the projector, and the computer-readablestorage medium,

the edge detection section may detect edges at a plurality of locationswithin a first sensed image based on the first sensing information, togenerate the first edge detection information, and

the projection-target area information generation section may generatethe provisional detection information by setting a linear approximatedstraight line or linear approximated curve, based on positionalinformation of the plurality of locations which is based on the firstedge detection information.

The image processing method may comprise:

detecting edges at a plurality of locations within a first sensed imagebased on the first sensing information, to generate the first edgedetection information; and

generating the provisional detection information by setting a linearapproximated straight line or linear approximated curve, based onpositional information of the plurality of locations which is based onthe first edge detection information.

This makes it possible for the image processing system and the like togenerate positional information for the projection-target area in ashorter time, by performing edge detection in a state in which theregion subjected to the edge detection processing has been reduced.

In the image processing system, the projector, and the computer-readablestorage medium,

the projection-target area detection section may comprise detectionpoint evaluation section which evaluates a plurality of edge detectionpoints, and

the detection point evaluation section may determine whether or not eachof the plurality of edge detection points is distanced from the linearapproximated straight line or the linear approximated curve by at leasta predetermined value, and may control the projection-target areainformation generation section in such a manner that a detection pointwhich is distanced by at least the predetermined value is excluded andthe linear approximated straight line or the linear approximated curveis reset.

The image processing method may comprise:

determining whether or not each of a plurality of edge detection pointsis distanced from the linear approximated straight line or the linearapproximated curve by at least a predetermined value; and

resetting the linear approximated straight line or the linearapproximated curve in such a manner that a detection point which isdistanced by at least the predetermined value is excluded.

This makes it possible for the image processing system and the like toreduce the effects of noise and generate positional information for theprojection-target area in a shorter time, by implementing processingthat excludes detection points that are separated from linearapproximated straight lines or the like.

The image processing system and the projector may further comprise:

a difference image generation section which generates a differenceimage; and

a center reference position detection section which detects a pluralityof center reference positions of a predetermined center block area inthe sensing area, based on the difference image,

wherein the projection-target area detection section may comprise searcharea determination section which sets an edge detection area by the edgedetection section,

wherein the image projection section may project a second calibrationimage,

wherein the sensing section may sense the projected second calibrationimage and may generate second sensing information,

wherein the difference image generation section may generate thedifference image, based on the first and second sensing information,

wherein the search area determination section may set the edge detectionarea outside the center block area,

wherein the first calibration image may be a monochrome calibrationimage, and

wherein the second calibration image may comprise the center block areawhich is smaller than the second calibration image and which ispositioned around the center of the second calibration image.

The computer-readable storage medium may cause a computer to functionas:

a difference image generation section which generates a differenceimage; and

a center reference position detection section which detects a pluralityof center reference positions of a predetermined center block area inthe sensing area, based on the difference image,

wherein the projection-target area detection section may comprise searcharea determination section which sets an edge detection area by the edgedetection section,

wherein the image projection section may project a second calibrationimage,

wherein the sensing section may sense the projected second calibrationimage and may generate second sensing information,

wherein the difference image generation section may generate thedifference image, based on the first and second sensing information,

wherein the search area determination section may set the edge detectionarea outside the center block area,

wherein the first calibration image may be a monochrome calibrationimage, and

wherein the second calibration image may comprise the center block areawhich is smaller than the second calibration image and which ispositioned around the center of the second calibration image.

The image processing method may further comprise:

projecting a second calibration image;

sensing the projected second calibration image by using the sensingsection, to generate second sensing information;

generating a difference image, based on the first and second sensinginformation;

detecting a plurality of center reference positions of a predeterminedcenter block area in the sensing area, based on the difference image;and

setting an edge detection area outside the center block area,

wherein the first calibration image may be a monochrome calibrationimage, and

wherein the second calibration image may comprise the center block areawhich is smaller than the second calibration image and which ispositioned around the center of the second calibration image.

This makes it possible for the image processing system and the like togenerate positional information for the projection-target area in ashorter time, by implementing processing that reduces the region that issubjected to edge detection.

In the image processing system and the projector,

the second calibration image may be configured of the center block area,a peripheral block area positioned on a periphery of the center blockarea, and a background area that is an area other than the center blockarea and the peripheral block area,

each pixel in the center block area and the peripheral block area mayhave a different index value from each pixel in the background area, and

the image processing system and the projector may comprise:

a peripheral reference position detection section which detects aplurality of peripheral reference positions of the peripheral block areain the sensing area, based on the center reference positions; and

a projection area information generation section which generatesprojection area information relating to a position of a projection areain the sensing area, based on the center reference positions and theperipheral reference positions.

In the computer-readable storage medium,

the second calibration image may be configured of the center block area,a peripheral block area positioned on a periphery of the center blockarea, and a background area that is an area other than the center blockarea and the peripheral block area,

each pixel in the center block area and the peripheral block area mayhave a different index value from each pixel in the background area, and

the computer-readable storage medium may cause a computer to functionas:

a peripheral reference position detection section which detects aplurality of peripheral reference positions of the peripheral block areain the sensing area, based on the center reference positions; and

a projection area information generation section which generatesprojection area information relating to a position of a projection areain the sensing area, based on the center reference positions and theperipheral reference positions.

In the image processing method,

the second calibration image may be configured of the center block area,a peripheral block area positioned on a periphery of the center blockarea, and a background area that is an area other than the center blockarea and the peripheral block area,

each pixel in the center block area and the peripheral block area mayhave a different index value from each pixel in the background area, and

the image processing method may further comprise:

detecting a plurality of peripheral reference positions of theperipheral block area in the sensing area, based on the center referencepositions; and

generating projection area information relating to a position of aprojection area in the sensing area, based on the center referencepositions and the peripheral reference positions.

With this embodiment, the image processing system and the like generatepositional information for the projection area by detecting a centerreference position of a center block area that is smaller than theprojection area corresponding to the projection image, even if part ofthe projection image is projected outside of the projection target.

In particular, since this embodiment enables the image processing systemand the like to determine the position of the projection area based notonly on a center reference position but also on a peripheral referenceposition of a peripheral block area that is positioned on the peripherythereof, the positional information of the projection area can begenerated more precisely.

In the image processing system, the projector, and the computer-readablestorage medium,

the projection area information generation section may generate theprojection area information by setting a plurality of approximatedstraight lines or approximated curves based on the center referencepositions and the peripheral reference positions and determining theshape or arrangement of the center block area and the peripheral blockarea.

The image processing method may comprise:

generating the projection area information by setting a plurality ofapproximated straight lines or approximated curves based on the centerreference positions and the peripheral reference positions anddetermining the shape or arrangement of the center block area and theperipheral block area.

In the image processing system, the projector, and the computer-readablestorage medium,

the projection area and the center block area may be rectangular areas,and

the projection area information generation section may determinepositions of four corners of the center block area by derivingintersections of the plurality of approximated straight lines orintersections of the plurality of approximated curves, and may generatethe projection area information indicating positions of four corners ofthe projection area, based on the positions of the four corners of thecenter block area.

In the image processing method,

the projection area and the center block area may be rectangular areas,and

the image processing method may comprise:

determining positions of four corners of the center block area byderiving intersections of the plurality of approximated straight linesor intersections of the plurality of approximated curves, and generatingthe projection area information indicating positions of four corners ofthe projection area, based on the positions of the four corners of thecenter block area.

Since this makes it possible for the image processing system and thelike to determine the positions of the four corners of the projectionarea based on the four corners of the center block area, the positionsof the four corners of the projection area can be determined with lessprocessing.

The image processing system and the projector may further comprise:

a projection-target area boundary point detection section which detectsa plurality of boundary points of the projection-target area, based onthe first sensing information and the center reference positions,

wherein the peripheral reference position detection section may detectthe peripheral reference positions which are positioned closer to theboundary points than the center reference positions, based on theboundary points.

The computer-readable storage medium may cause a computer to functionsas:

a projection-target area boundary point detection section which detectsa plurality of boundary points of the projection-target area, based onthe first sensing information and the center reference positions,

wherein the peripheral reference position detection section may detectthe peripheral reference positions, based on the boundary points.

The image processing method may comprise:

detecting a plurality of boundary points of the projection-target area,based on the first sensing information and the center referencepositions; and

detecting the peripheral reference positions, based on the boundarypoints.

Note that in this case, the peripheral reference position detectionsection and the image processing method could detect a peripheralreference position that is positioned closer to the boundary point thanthe center reference position.

Since this makes it possible for the image processing system and thelike to base the generation of positional information for the projectionarea on a plurality of reference position that are separated, there isless room for error and the positional information of the projectionarea can be generated more precisely.

The image processing system and the projector may further comprise:

an image distortion correction section which determines whether there isdistortion in an image projected by the image projection section, basedon the projection-target area information and the projection areainformation, and correcting an image signal to correct the distortion,

wherein the image projection section may project an image based on theimage signal which has been corrected by the image distortion correctionsection.

The computer-readable storage medium may cause a computer to functionas:

an image distortion correction section which determines whether there isdistortion in an image projected by the image projection section, basedon the projection-target area information and the projection areainformation, and correcting an image signal to correct the distortion,

wherein the image projection section may project an image based on theimage signal which has been corrected by the image distortion correctionsection.

The image processing method may comprise:

determining whether there is distortion in an image to be projected,based on the projection-target area information and the projection areainformation, and correcting an image signal to correct the distortion.

Since this makes it possible for the image processing system and thelike to generate positional information for each of theprojection-target area and the projection area with less processing thanin the art, image distortion can be corrected more efficiently than inthe art.

The present invention is described below as being applied to a projectorhaving an image processing system, with reference to the accompanyingfigures. Note that the embodiments described below do not limit thescope of the invention defined by the claims laid out herein. Inaddition, not all of the elements of the embodiments described belowshould be taken as essential requirements of the present invention.

First Embodiment

A schematic view of an image projection state of a first embodiment ofthe present invention is shown in FIG. 1. A conceptual view of thepositional relationship of a screen 10 and a projection image 12 inaccordance with the first embodiment is shown in FIG. 2.

A projector 20 projects an image towards the screen 10. This causes thedisplay of the projection image 12 on the screen 10.

The projector 20 of the first embodiment has a sensor 60 that is asensing means. The sensor 60 senses the screen 10 on which theprojection image 12 is displayed, through a sensing screen, andgenerates sensing information. The projector 20 adjusts the distortionand the display position of the projection image 12, based on thesensing information.

However, if part of the projection image 12 is displayed outside of thescreen 10, as shown by way of example in FIG. 2, a prior-art projectorwould be unable to adjust the distortion of the projection image 12 andthe display position thereof, based on sensing information.

This is because the screen 10 and the wall that is behind the screen 10are separated by some distance and the prior-art projector is unable toconvert the position of the peak point of the projection image 12, whichis either displayed on the wall or background objects at unknowndistances or is not displayed at all, into a position on the plane ofthe screen 10, even though the projection image 12 is within the sensingarea of the sensor 60.

The projector 20 of the first embodiment uses a calibration image thatdiffers from that of the prior art in that it enables the determinationof the position of the projection image 12 more precisely under a widerrange of conditions than in the prior art, by implementing simple searchprocessing based on sensing information of that calibration image.

The description now turns to function blocks of the projector 20 forimplementing the above functions.

A functional block diagram of the projector 20 in accordance with thefirst embodiment is shown in FIG. 3.

The projector 20 comprises an input signal processing section 110 thatconverts analog RGB signals (R1, G1, and B1) into digital RGB signals(R2, G2, and B2); a color conversion section 120 that converts thosedigital RGB signals (R2, G2, and B2) into digital RGB signals (R3, G3,and B3), in order to correct the color and brightness of the image; anoutput signal processing section 130 that converts those digital RGBsignals (R3, G3, and B3) into analog RGB signals (R4, G4, and B4); andan image projection section 190 that projects an image based on thoseanalog RGB signals.

The image projection section 190 comprises a spatial light modulator192, a drive section 194 for driving the spatial light modulator 192, alight source 196, and a lens 198. The drive section 194 drives thespatial light modulator 192, based on image signals from the outputsignal processing section 130. The image projection section 190 projectslight from the light source 196 through the spatial light modulator 192and the lens 198.

The projector 20 also comprises a calibration information generationsection 172 that generates calibration information for displaying firstand second calibration images; the sensor 60 that generates sensinginformation for the calibration images; and a sensing informationstorage section 140 that temporarily stores sensing information from thesensor 60.

The projector 20 further comprises a projection area detection section150 that detects the position of the projection area in the sensingscreen (sensing area) of the sensor 60, based on the sensinginformation. The projection area detection section 150 comprises adifference image generation section 152 that generates a differenceimage between a first sensed image and a second sensed image, a centerreference position detection section 154 that detects a plurality ofcenter reference positions of a center block area comprised within thedifference image, a peripheral reference position detection section 156that detects a plurality of peripheral reference positions of peripheralblock areas comprised within the difference image, and a projection areainformation generation section 158 that generates projection areainformation indicating the position of the projection area, based on thereference positions.

The projector 20 also has a function that corrects distortion of theprojection image 12. To implement this function, the projector 20comprises a luminance peak position detection section 164 that detects aluminance peak position (the position of the pixel at which theluminance value is a maximum) within the projection area, based on theprojection area information; an image distortion correction amountcalculation section 162 that calculates the amount of image distortioncorrection, based on the luminance peak position; and an imagedistortion correction section 112 that corrects the image signal, basedon the image distortion correction amount.

The hardware described below can be applied for implementing thefunctions of the above-described components of the projector 20.

A block diagram of hardware for the projector 20 of the first embodimentis shown in FIG. 4.

For example, the configuration could be implemented by using an A/Dconverter 930 and image processing circuit 970, or the like, as theinput signal processing section 110; RAM 950 or the like as the sensinginformation storage section 140; the image processing circuit 970 or thelike as the projection area detection section 150 and the luminance peakposition detection section 164; a CPU 910 or the like as the imagedistortion correction amount calculation section 162; the imageprocessing circuit 970 and the RAM 950, or the like, as the calibrationinformation generation section 172; a D/A converter 940 or the like asthe output signal processing section 130; a liquid-crystal panel 920 orthe like as the spatial light modulator 192; and a ROM 960 or the likewhich stores a liquid-crystal light valve driver for driving theliquid-crystal panel 920, as the drive section 194.

Note that these components can exchange information between themselvesover a system bus 980.

These components could also be implemented in a hardware fashion bycircuitry, or in a software manner by drivers or the like.

In addition, an information storage medium 900 which stores a programthat causes a computer to function as components such as the differenceimage generation section 152 could be installed in the computer, and thecomputer reads out the program in order to function as the differenceimage generation section 152, etc.

This information storage medium 900 could be a CD-ROM, DVD-ROM, ROM,RAM, or HDD, by way of example, and the method of reading the programtherefrom could be a direct method or an indirect method.

The description now turns to the flow of projection area positiondetection processing that is performed by using these components.

A flowchart of position detection processing for the projection area inaccordance with the first embodiment is shown in FIG. 5. A conceptualview of a first calibration image 13 is shown in FIG. 6A and aconceptual view of a second calibration image 14 is shown in FIG. 6B.

The projector 20 first projects an all-white calibration image (in whichthe entire image is white), as shown in FIG. 6A, as the firstcalibration image 13 (step S1). More specifically, the calibrationinformation generation section 172 generates calibration information(such as RGB signals) for the first calibration image 13 and the imageprojection section 190 projects an all-white calibration image based onthat calibration information.

The sensor 60 senses the first calibration image 13 at an automaticexposure setting, and generates first sensing information (step S2). Thesensing information storage section 140 stores the first sensinginformation.

The projector 20 then projects the second calibration image 14 shown inFIG. 6B as the second calibration image 14 (step S3). More specifically,the calibration information generation section 172 generates calibrationinformation for the second calibration image 14 and the image projectionsection 190 projects the second calibration image 14 based on thatcalibration information.

In this embodiment, the second calibration image 14 is a pattern imageof a checker pattern such that, if the entire image is divided into nineequal blocks, the center block and the four peripheral block areas atthe four corners are black and the remaining block areas are white.

The sensor 60 senses the second calibration image 14 on the screen 10 atthe same exposure as that for the first calibration image 13, andgenerates second sensing information (step S4). The sensing informationstorage section 140 stores the second sensing information.

The difference image generation section 152 generates a difference imagebetween the first calibration image 13 and the second calibration image14, based on the first and second sensing information (step S5). Notethat this difference image is an image obtained by calculatingdifferences such as those of luminance values for each pixel, by way ofexample. This difference image could be an image having a differencevalue for each pixel if that difference value exceeds a predeterminedthreshold at that pixel, but the value is zero at all other pixelpositions, by way of example. Note that the difference image generationsection 152 need not necessarily calculate differences for the entireimage; it could also calculate differences only within a area (a portionof the image) that is necessary for the processing described below.

After the difference image is generated, the projection area detectionsection 150 detects a plurality of center reference positions (four inthis embodiment) of the center block area comprised within thedifference image and a plurality of peripheral reference positions(eight in this embodiment) comprised within the difference image.

A conceptual view of a search method in a first stage in the detectionof the center reference positions in the first embodiment is shown inFIG. 7. A conceptual view of a search method in a second stage in thedetection of the center reference positions in the first embodiment isshown in FIG. 8.

The center reference position detection section 154 first detects thefour center reference positions of the pattern image, in order to detectthe position of the projection area (the area corresponding to theprojection image 12) in a sensing area 15 corresponding to the sensingscreen (step S6). Note that a projection-target area (screen area) 18 isshown in these figures to facilitate comprehension of the description,but it is possible that the projection-target area 18 and some ofperipheral block areas 17-1 to 17-4 that are outside of theprojection-target area 18 will not exist in the difference image inpractice.

More specifically, the center reference position detection section 154identifies points P1 and P2 at which difference values change, bysearching the difference values in the difference image for each pixelfrom y=yp to y=ym along a vertical line x=xc that is assumed to bepositioned on a center block area 16, as shown in FIG. 7. Assume, by wayof example that the coordinates of these two points are P1 (xc, y1) andP2 (xc, y2).

Note that values such as xc, yp, and ym for the search referenceposition could be determined by the angle of view and position of eachof the lens 198 and the sensor 60, or they could be determined byexperimentation, or they could be determined based on experimentation,by way of example. This also concerns other search reference positionsused later in this description.

The center reference position detection section 154 then identifiespoints P4 and P3 at which difference values change, by searching thedifference values in the difference image for each pixel from x=xm tox=xp along a horizontal line y=yc referenced to P1 and P2, as shown inFIG. 8. Note that in this case yc=(y1+y2)/2, by way of example.

Thus the center reference position detection section 154 outputs centerreference positional information indicating the four center referencepositions P1 (xc, y1), P2 (xc, y2), P3 (x1, yc), and P4 (x2, yc) of thecenter block area 16 to the peripheral reference position detectionsection 156.

The peripheral reference position detection section 156 detects theeight peripheral reference positions of the pattern image, based on thecenter reference positional information (step S7).

A conceptual view of a search method in a first stage in the detectionof the peripheral reference positions in accordance with the firstembodiment is shown in FIG. 9. Similarly, a conceptual view of a searchmethod in a second stage in the detection of the peripheral referencepositions in accordance with the first embodiment is shown in FIG. 10.

More specifically, the peripheral reference position detection section156 searches for a point at which there is a change in difference valuein the pixels of the difference image, along the line y=yh that is m %above the y-coordinate y1 of P1, in the positive direction of the x-axisfrom the x-coordinate xh that is a few percent on the center side fromthe x-coordinate x1 of P3. This identifies a point P5 at which thedifference values change.

Similarly, the peripheral reference position detection section 156searches for a point at which there is a change in difference value inthe pixels of the difference image, along the line y=yn that is m %below the y-coordinate y2 of P2, in the positive direction of the x-axisfrom the x-coordinate xh. This identifies a point P6 at which thedifference values change.

By a similar method, points P7 to P12 are identified, as shown in FIG.10. The peripheral reference position detection section 156 then outputsperipheral reference positional information indicating the coordinatesof these eight points and center reference positional information to theprojection area information generation section 158.

The projection area information generation section 158 uses approximatedstraight lines (or approximated curves) based on the peripheralreference positional information and the center reference positionalinformation, to detect the positions of the four corners of theprojection area (step S8).

A conceptual view of a first stage of setting approximated straightlines in accordance with the first embodiment is shown in FIG. 11. Aconceptual view of a second stage of setting approximated straight linesin accordance with the first embodiment is shown in FIG. 12.

The projection area information generation section 158 sets anapproximated straight line, as shown by the broken line in FIG. 11,based on the coordinates of points P5, P3, and P6. By a similar method,the projection area information generation section 158 sets fourapproximated straight lines, as shown by the broken lines in FIG. 12,and identifies four intersections A (xA, yA) to D (xD, yD) between theseapproximated straight lines as the four corners of the center block area16.

Since the center block area 16 has an area corresponding to an imagethat is shrunk to 1/9th of the original projection image 12, thedescription below assumes that the points EFGH of the four corners ofthe projection area correspond to the projection image 12. In otherwords, E (xE, yE)=(2*xA−xC, 2*yA−yc), F (xF, yF)=(2*xB−xD, 2*yB−yD), G(xG, yG)=(2*xC−xA, 2*yC−yA), and H (xH, yH)=(2*xD−xB, 2*yD−yB).

As described above, this embodiment can detect the positions of the fourcorners of the projection area in the sensing area 15, inevitably whenthe projection image 12 is comprised within the screen 10, but also whenpart of the projection image 12 is displayed outside the screen 10. Ofcourse it is also possible for the projector 20 to convert thepositional information of the projection area into the plane of thescreen 10, to generate positional information for the four corners ofthe projection image 12.

This enables the projector 20 to correct distortion of the projectionimage 12 or adjust the position thereof, and also perform suitabledetection of a position that is indicated by using a laser pointer orthe like within the projection image 12.

If the projector 20 performs correction (known as keystone correction)of the projection image 12, by way of example, the luminance peakposition detection section 164 is used to detect the luminance peakposition at which the luminance value is at a maximum within theprojection area within the sensing area, based on the sensinginformation for the first calibration image 13 and the projection areainformation that indicates the four corners of the projection area fromthe projection area information generation section 158.

If the screen 10 and the projector 20 are face-on to each other, forexample, the center of the projection area would be the luminance peakposition. Alternatively, if the luminance values on the left side of theprojection area are high, it is possible to determine that the opticalaxis of the projection has slipped toward the left from the center ofthe projection image 12, and thus it is possible to determined that theprojection image 12 is deformed into a lozenge shape in which the leftside is shorter and the right side longer. Thus the distortion of theimage can be determined by determining the luminance peak positionwithin the projection area.

The image distortion correction amount calculation section 162calculates a correction amount corresponding to the image distortion,based on the luminance peak position within the projection area.

The image distortion correction section 112 within the input signalprocessing section 110 then corrects the input image signal in such amanner that the distortion of the image is corrected, based on thatcorrection amount.

With the above sequence, the projector 20 can correct distortion of theimage, even when part of the projection image 12 is displayed outside ofthe screen 10. It should be obvious to those skilled in the art that themethod of correcting image distortion is not limited to this method. Forexample, the projector 20 could detect the pixel that has the greatestluminance value within the sensed image and base the correction ofdistortion of that image on the position of that pixel.

The projector 20 could also identify the four corners of the projectionarea to a higher precision than that obtained by using a pattern imagethat has a feature only at the center, by using an image that hasfeatures at the periphery thereof in addition to the center, such as thepattern image shown in FIG. 6B.

For example, during the identification of the points P1 and P2 in FIG.7, the projector 20 could also identify points at which the luminancevalue in the vicinity thereof changes. However, when an approximatedstraight line is set by using a plurality of points at such a narrowspacing, an error of one pixel in a point that is the origin of theapproximated straight line can have a greater effect than in anapproximated straight line formed by using a plurality of points at awider spacing.

Since this embodiment enables the projector 20 to set approximatedstraight lines by using a plurality of points at wider spacing, by usingthe reference points of the center block area 16 and the referencepoints of the peripheral block areas 17-1 to 17-4, it is possible toidentify the four corners of the projection area at a higher precision.

This enables the projector 20 to determine the position of the entireprojection area very precisely, avoiding the effects of shading of theprojector 20 or the sensor 60.

In addition, this embodiment enables the projector 20 to detect theposition of the projection area more easily and also more rapidly, bysearching only the necessary areas of the difference image, not theentire difference image.

The first sensing information can be generated at an exposure suited tothe usage environment by sensing the all-white image at a temporaryautomatic exposure setting and generating the corresponding firstsensing information, during the projection of the calibration images.The projector 20 can also generate the second sensing information at anexposure suited to the generation of the difference image, by generatingthe second sensing information at the exposure used during the sensingof the all-white image.

In particular, by sensing an all-white image at the automatic exposuresetting, the sensor 60 can utilize the dynamic range of the sensor 60more effectively than in a method of sensing images at a fixed exposure,even when the screen 10 is affected by ambient light, when thereflection of the projected light is too weak because the projectiondistance is too great or the reflectivity of the screen 10 is too low,or when reflection of the projected light is too strong because theprojection distance is too close or the reflectivity of the screen 10 istoo high.

Modification of First Embodiment

The description above concerned a first embodiment of the presentinvention, but the application of the present invention is not limitedto the above-described embodiment.

For example, the searching can be done in any sequence, so that theprojector 20 could first search in the lateral direction with respect tothe difference image to detect a center reference position or aperipheral reference position, then base a search in the verticaldirection on that center reference position or that peripheral referenceposition.

Similarly, the projector 20 could perform various different processingsby using the positional information for the projection area, such ascorrecting color variations within the projection area or detecting anindicated position within the projection area, based on the projectionarea information, other than the correction of image distortion based onthe projection area information.

The projector 20 could also detect the projection area after detectingthe projection-target area 18.

A functional block diagram of the projector 20 in accordance with amodification of the first embodiment is shown in FIG. 13. A conceptualview of a search method in a first stage in the detection of theperipheral reference positions in this modification of the firstembodiment is shown in FIG. 14. Similarly, a conceptual view of a searchmethod in a second stage in the detection of the peripheral referencepositions in this modification of the first embodiment is shown in FIG.15.

A projection-target area boundary point detection section 159 isprovided within the projection area detection section 150, as shown byway of example in FIG. 13.

The center reference position detection section 154 outputs centerreference positional information to the projection-target area boundarypoint detection section 159.

The projection-target area boundary point detection section 159 searchesthe first sensed image towards the outer sides of the center block area16 from intersections between lines each of which is positioned a fewpercent within the center block area 16 from each of the points P3 andP4 and each of the lines y=y1 and y=y2, to detect edges with respect topixels on the lines each of which is positioned a few percent within thecenter block area 16 from each of the points P3 and P4, as shown in FIG.14. Note that a generic method is used in the edge detection. Thisidentifies points T, U, V, and W, as shown in FIG. 14. Theprojection-target area boundary point detection section 159 outputsprojection-target area boundary point information indicating thepositions of the points T, U, V, and W to the peripheral referenceposition detection section 156.

The peripheral reference position detection section 156 detects aposition Y=yQ to act as reference for a search in the lateral directionon the upper side, based on whichever is the smaller value of yT that isthe Y-coordinate of the point T and yU that is the Y-coordinate of thepoint U, together with y1 that is the Y-coordinate of the point P1.Similarly, the peripheral reference position detection section 156detects a position Y=yR to act as reference for a search in the lateraldirection on the lower side, based on whichever is the smaller value ofyV that is the Y-coordinate of the point V and yW that is theY-coordinate of the point W, together with y2 that is the Y-coordinateof the point P2.

The peripheral reference position detection section 156 identifies fourpoints P5 to P8 by searching outward on the lines Y=yQ and Y=yR in thedifference image from each of the intersections between the fourstraight lines X=xt, X=xU, Y=yQ, and Y=yR, and detecting pixels that areoutput. The peripheral reference position detection section 156identifies the remaining four points P9 to P12 by a similar method.

The projection area detection section 150 can identify the centerreference positions of the center block area 16 and the peripheralreference positions of the peripheral block areas 17-1 to 17-4 by such amethod, to identify the positions of the four corners of the projectionarea.

In particular, such a method enables the projection area informationgeneration section 158 to avoid unwanted processing for detectingperipheral reference positions outside the projection-target area, incomparison with the previously described method, and also enables it toobtain an approximated straight line in a state in which the threepoints for deriving that approximated straight line are further apart.This enables the projector 20 to detect the position of the projectionarea to a higher precision.

Second Embodiment

The projector 20 in accordance with the second embodiment determines thepositional relationship between the screen 10 and the projection image12 and also the shape of the projection image 12, based on sensinginformation, and adjusts distortion of the projection image 12 and thedisplay position thereof.

The projector 20 of the second embodiment generates positionalinformation for a projection-target area (an area corresponding to thescreen 10) in a projection-target area within the sensing area of thesensor 60 and a projection area (an area corresponding to the projectionimage 12), within a shorter time and also more accurately, by performingimage processing that differs from that in the prior art.

The description now turns to function blocks of the projector 20 forimplementing such functions.

A functional block diagram of the projector 20 of the second embodimentis shown in FIG. 16.

The projector 20 comprises the input signal processing section 110 thatconverts analog RGB signals (R1, G1, and B1) that are input from apersonal computer (PC) or the like, into digital RGB signals (R2, G2,and B2); the color conversion section 120 that converts those digitalRGB signals (R2, G2, and B2) into digital RGB signals (R3, G3, and B3),in order to correct the color and brightness of the image; the outputsignal processing section 130 that converts those digital RGB signals(R3, G3, and B3) into analog RGB signals (R4, G4, and B4); and the imageprojection section 190 that projects an image based on those analog RGBsignals.

The image projection section 190 comprises the spatial light modulator192, the drive section 194 for driving the spatial light modulator 192,the light source 196, and the lens 198. The drive section 194 drives thespatial light modulator 192, based on analog RGB signals from the outputsignal processing section 130. The image projection section 190 projectslight from the light source 196 through the spatial light modulator 192and the lens 198.

The projector 20 also comprises the calibration information generationsection 172 that generates calibration information for displaying firstand second calibration images; the sensor 60 that generates sensinginformation for the calibration images; and the sensing informationstorage section 140 that temporarily stores sensing information from thesensor 60.

The projector 20 further comprises the projection area detection section150 that detects the position of the projection area in the sensingscreen (sensing area) of the sensor 60, based on the sensinginformation. The projection area detection section 150 comprises thedifference image generation section 152 that generates a differenceimage between the first sensed image and the second sensed image, thecenter reference position detection section 154 that detects a pluralityof center reference positions of the center block area comprised withinthe difference image, the peripheral reference position detectionsection 156 that detects a plurality of peripheral reference positionsof peripheral block areas comprised within the difference image, and theprojection area information generation section 158 that generatesprojection area information indicating the position of the projectionarea, based on the reference positions.

The projector 20 also comprises a projection-target area detectionsection 180 that generates projection-target area information relatingto the position of the projection-target area that corresponds to thescreen 10 in the sensing area of the sensor 60. The projection-targetarea detection section 180 comprises a search area determination section182 that sets an edge detection area, an edge detection section 184, adetection point evaluation section 188 that evaluates edge detectionpoints, and a projection-target area information generation section 186that generates provisional detection information from a provisionaldetection of the projection-target area and also generatesprojection-target area information.

The projector 20 also has image distortion correction means thatcorrects distortion of the projection image 12. More specifically, theprojector 20 has the image distortion correction amount calculationsection 162 that calculates the amount of image distortion correction,based on the projection area information, the projection-target areainformation, and the projectable area information; and the imagedistortion correction section 112 that corrects the image signal, basedon that image distortion correction amount; as the image distortioncorrection means.

The hardware for implementing the functions of the above-describedprojector 20 could be as described below with reference to FIG. 4, byway of example.

For example, these functions could be implemented by using the A/Dconverter 930 and the image processing circuit 970, or the like, as theinput signal processing section 110; the RAM 950 or the like as thesensing information storage section 140; the image processing circuit970 or the like as the projection area detection section 150 and theluminance peak position detection section 164; the CPU 910 or the likeas the image distortion correction amount calculation section 162; theimage processing circuit 970 and the RAM 950, or the like, as thecalibration information generation section 172; the D/A converter 940 orthe like as the output signal processing section 130; the liquid-crystalpanel 920 or the like as the spatial light modulator 192; and the ROM960 or the like which stores a liquid-crystal light valve driver fordriving the liquid-crystal panel 920, as the drive section 194.

Note that these components can exchange information between themselvesover the system bus 980.

These components could also be implemented in a hardware fashion bycircuitry, or in a software manner by drivers or the like.

In addition, the information storage medium 900 which stores a programthat causes a computer to function as components such as theprojection-target area information generation section 186 could beinstalled in the computer, and the computer reads out the program inorder to function as the projection-target area information generationsection 186, etc.

This information storage medium 900 could be a CD-ROM, DVD-ROM, ROM,RAM, or HDD, by way of example, and the method of reading the programtherefrom could be a direct method or an indirect method.

Instead of the information storage medium 900, it is also possible todownload a program that implements the above-described functions, from ahost device through a transfer path, in order to implement theabove-described functions.

The description now turns to the projection-target area positiondetection processing that is performed by using the above-describedcomponents. Note that the projection area position detection processingis similar to that of the first embodiment so further descriptionthereof is omitted. This second embodiment differs from the firstembodiment shown in FIG. 7 in that it is assumed that the entireprojection-target area 18 does not protrude from the peripheral blockareas 17-1 to 17-4.

The description below relates to the flow of position detectionprocessing for the projection-target area 18.

A flowchart of the position detection processing for theprojection-target area 18 in accordance with the second embodiment isshown in FIG. 17.

In the processing of step S2 described above, the sensor 60 senses thefirst calibration image 13 at a resolution (such as VGA) that is lowerthan a predetermined resolution (such as SVGA), to generate the firstsensing information (step S2). The sensing information storage section140 stores the first sensing information.

While the first calibration image 13 is still projected onto the screen10, the sensor 60 then senses the first calibration image 13 at aresolution (such as XGA, SXGA, or UXGA) that is higher than theabove-described low resolution, to generate the third sensinginformation (step S1). The sensing information storage section 140stores the third sensing information.

The projection-target area detection section 180 generates theprojection-target area information that indicates positional informationof the projection-target area 18, based on the first and third sensinginformation.

A conceptual view of a search method in a first stage of the detectionof the projection-target area 18 in accordance with the secondembodiment is shown in FIG. 18. A conceptual view of a search method ina second stage of the detection of the projection-target area 18 inaccordance with the second embodiment is shown in FIG. 19. Similarly, aconceptual view of a search method in a third stage of the detection ofthe projection-target area 18 in accordance with the second embodimentis shown in FIG. 20. Finally, a conceptual view of edge detection pointevaluation processing in accordance with the second embodiment is shownin FIG. 21.

First of all, the search area determination section 182 sets four searchsupport lines in the sensing area, as indicated by broken lines in FIG.18, based on the first sensing information and coordinates informationfor the four corners ABCD of the above-described center block area 16,to determining the candidates for edge detection (step S12). Morespecifically, the search area determination section 182 targets awhite-colored sensed image that has been sensed at a low resolution, andsets search support lines that are p % further outward from thecoordinates of the four corners ABCD of the center block area 16detected by the center reference position detection section 154.

For example, the first search support line is given by y=round[{max(yA,yB)+(yA−yD)*p/100}*a], the second search support line is given byx=round[{max (xB,xC)+(xB−xA)*p/100}*a], the third search support line isgiven by y=round[{min (yC,yD)−(yA−yD)*p/100}*a], and the fourth searchsupport line is given by x=round[{min (xA,xD)−(xB−xA)*p/100}*a]. Notethat the terms “max,” “min”, “round”, and “a” in this case are afunction that returns the maximum value from among arguments, a functionthat returns the minimum value from among arguments, a function thatreturns an integer in which the first place after the decimal point isrounded off, and a coefficient for converting the resolution (highresolution) of the second sensed image into the resolution (lowresolution) of the first sensed image. Note that “a” is not necessary ifthere is no need to convert the resolution.

Setting the four search support lines in this manner makes it possibleto determine four intersections IJKL of the four search lines, as shownin FIG. 18.

The edge detection section 184 performs edge detection one pixel at atime, on the first sensed image along the search lines on the searchsupport lines, towards the outer sides of the areas IJKL from thecorresponding intersections IJKL (step S13). This enables the detectionof eight edge detection points MNOPQRST, as shown in FIG. 18.

In addition, the edge detection section 184 performs edge detection inone-pixel increments in the direction of the boundary line of theprojection-target area 18 from a line segment TO, the direction of theboundary line of the projection-target area 18 from a line segment NQ,the direction of the boundary line of the projection-target area 18 froma line segment PS, and the direction of the boundary line of theprojection-target area 18 from a line segment RM.

In this case, the description concerns the example of edge detection inthe direction of the boundary line of the projection-target area 18 fromthe line segment TO. The edge detection section 184 sets the area foredge detection for the line segment TO as seven search lines in a linesegment IJ and two each in line segments TI and JO, parallel to theY-axis and in the positive direction, as shown in FIG. 19. Note that thearea in which the seven search lines are set is called the inner-sidesearch area and the two areas in which two search lines are set arecalled the outer-side search areas.

The edge detection section 184 can therefore detect a maximum of elevenedge detection points (thirteen, including the points M and N) on thestraight line MN, by performing edge detection one pixel at a time alongthese search lines. The edge detection section 184 performs similar edgedetection for the other line segments NQ, PS, and RM.

Note that if the edge detection section 184 is unable to detect onepoint of a pair of edge detection points MN, OP, QR, and ST within thesensing area 15, it assumes that there is no boundary line for theprojection-target area 18 in the outer-side search area and thus it doesnot set search lines and detect edges within that area. In addition, ifthe edge detection section 184 is unable to detect both points of a pairof edge detection points MN, OP, QR, and ST within the sensing area 15,it assumes that there is no boundary line for the projection-target area18 in the vicinity of the direction parallel to the line segment that itwas unable to detect, and thus it does not set search lines and detectedges in order to search for the line segment it cannot detect, withinthe inner-side search area and outer-side search areas.

The implementation of this processing enables the edge detection section184 to omit edge detection in areas where the probability of theexistence of the projection-target area 18 is low, making it possible tospeed up the processing.

The projection-target area information generation section 186 determinesthe projection-target area 18 provisionally by setting a linearapproximated straight line or linear approximated curve, such as thatshown by the broken line in FIG. 20, based on the plurality of edgedetection points detected by the edge detection section 184 (step S14).

The detection point evaluation section 188 also evaluates each edgedetection point by determining whether or not each of the plurality ofedge detection points detected by the edge detection section 184 isseparated by at least a predetermined value from the linear approximatedstraight line or linear approximated curve set by the projection-targetarea information generation section 186 (step S15).

It could happen that illumination light 19 is comprised within thesensed image, as shown in FIG. 21, so that part of the illuminationlight is detected during the edge detection. If that should happen, theedge detection point T shown in FIG. 21 would be excluded from theprocessing performed by the projection-target area informationgeneration section 186 because it would be separated by at least thepredetermined value from the boundary line of the projection-target area18.

In this manner, the projection-target area information generationsection 186 detects the projection-target area 18 at a higher precision,by using only the edge detection points that are not excluded from theprocessing.

More specifically, the edge detection section 184 performs the edgedetection on pixels peripheral to the edge detection points that havenot been excluded from the processing, based on the third sensinginformation that is high-resolution sensing information (step S16). Theedge detection section 184 outputs the edge detection information to theprojection-target area information generation section 186.

The projection-target area information generation section 186 determinesthe projection-target area 18 by setting the linear approximatedstraight line or linear approximated curve again, based on that edgedetection information (step S17). The projection-target area informationgeneration section 186 generates projection-target area informationindicating the positions of the four corners of the projection-targetarea 18.

As described above, this embodiment enables the projector 20 to generatepositional information of the projection-target area 18 in a shortertime and also more accurately, by detecting the projection-target area18 based on the high-resolution sensing information in the vicinity ofboundary lines of the projection-target area 18 that have been detectedprovisionally based on the low-resolution sensing information. Thisenables the projector 20 to reduce the amount of calculations performedby the entire image processing system and thus execute image processingrapidly at a low load.

This embodiment also enables the projector 20 to generate positionalinformation of the projection-target area 18 within a shorter time, byperforming edge detection in a state in which the region subjected tothe edge detection processing has been reduced.

The projector 20 can reduce the effects of noise and generate thepositional information of the projection-target area 18 accurately, byexcluding edge detection points that are separated from linearapproximated straight lines or the like, from the processing. Theprojector 20 can also avoid erroneous detection of edges that couldoccur in the projection image 12 and thus perform more precise edgedetection, by using a calibration image during the edge detection thatdoes not comprise high-frequency components, such as an all-white image.

This enables the projector 20 to correct distortion of the projectionimage 12 or adjust the position thereof, and also perform suitabledetection of a position that is indicated by using a laser pointer orthe like within the projection image 12.

In this embodiment, the image distortion correction amount calculationsection 162 determines the positional relationship between the screen 10and the projection image 12, based on projection area information fromthe projection area information generation section 158 andprojection-target area information from the projection-target areainformation generation section 186, and not only corrects distortion ofthe projection image 12 but also calculates image distortion correctionamounts that ensure that the projection image 12 has a desired aspectratio (the ratio of vertical to horizontal).

The image distortion correction section 112 corrects the image signals(R1, G1, and B1), based on those image distortion correction amounts.This ensures that the projector 20 can project an image withoutdistortion, in a form in which the desired aspect ratio is maintained.

The method used for correcting image distortion is not limited to thismethod. For example, the projector 20 could detect the pixel that hasthe maximum luminance value within the sensed image, and base thecorrection of distortion of the image on the position of that pixel.

The projector 20 could also identify the four corners of the projectionarea to a higher precision than that obtained by using a pattern imagethat has a feature only at the center, by using an image that hasfeatures at the periphery thereof in addition to the center, such as thepattern image shown in FIG. 6B.

For example, during the identification of the points P1 and P2 in FIG.7, the projector 20 could also identify points at which the luminancevalue in the vicinity thereof changes. However, when an approximatedstraight line is set by using a plurality of points at such a narrowspacing, an error of one pixel in a point that is the origin of theapproximated straight line can have a greater effect than in anapproximated straight line formed by using a plurality of points at awider spacing.

Since this embodiment enables the projector 20 to set approximatedstraight lines by using a plurality of points at a wider spacing, byusing the reference points of the center block area 16 and the referencepoints of the peripheral block areas 17-1 to 17-4, it is possible toidentify the four corners of the projection area to a higher precision.

This enables the projector 20 to determine the position of the entireprojection area very precisely, avoiding the effects of shading of theprojector 20 or the sensor 60.

In addition, this embodiment enables the projector 20 to detect theposition of the projection area more simply and also rapidly, bysearching only the necessary areas of the difference image, not theentire difference image.

The first sensing information can be generated at an exposure suited tothe usage environment by sensing the all-white image at a temporaryautomatic exposure setting and generating the first sensing informationtherefrom, during the projection of the calibration images. Theprojector 20 can also generate the second sensing information at anexposure suited to the generation of the difference image, by generatingthe second sensing information at the exposure used during the sensingof the all-white image.

In particular, by sensing an all-white image at the automatic exposuresetting, the sensor 60 can utilize the dynamic range of the sensor 60more effectively than in a method of sensing images at a fixed exposure,even when the screen 10 is affected by external light, when thereflection of the projected light is too weak because the projectiondistance is too great or the reflectivity of the screen 10 is too low,or when reflection of the projected light is too strong because theprojection distance is too close or the reflectivity of the screen 10 istoo high.

Modification of Second Embodiment

The description above concerned preferred embodiment of the presentinvention, but the application of the present invention is not limitedto the embodiment described above.

For example, when the first sensing information and the third sensinginformation are generated, the first sensing information and the thirdsensing information could be generated the first time by the sensor 60sensing the first calibration image 13 at the high resolution thensubjecting the high-resolution sensing information to image processingto convert it into low-resolution sensing information.

An angle of view adjustment function (zoom function) of the projector 20could be used for adjustments of the position or size of the projectionimage 12. This would enable reliable detection of the projection-targetarea under darkroom conditions.

In addition, the searching can be done in any sequence, so that theprojector 20 could first search in the lateral direction with respect tothe difference image to detect a center reference position or aperipheral reference position, then base a search in the verticaldirection on that center reference position or that peripheral referenceposition.

Similarly, the projector 20 could perform various different processingsby using the positional information for the projection area, such ascorrecting color variations within the projection area or detecting anindicated position within the projection area, based on the projectionarea information, other than the correction of image distortion based onthe projection area information.

The projector 20 could also detect the projection area after detectingthe projection-target area 18. For example, a projection-target areaboundary point detection section could be provided to detect a pluralityof boundary points of the projection-target area 18, based on the firstsensing information and the center reference position. In addition, theperipheral reference position detection section 156 could be configuredto detect peripheral reference positions that are positions closer tothose boundary points than the center reference position, based on thoseboundary points.

A conceptual view of a search method in a first stage in the detectionof peripheral reference positions in accordance with a modification ofthe second embodiment is shown in FIG. 22. A conceptual view of a searchmethod in a second stage in the detection of the peripheral referencepositions in accordance with this modification of the second embodimentis shown in FIG. 23.

For example, the projection-target area boundary point detection sectioncould perform the edge detection one pixel at a time outward from thepoints y1 and y2, which are the Y-coordinates of P1 and P2, on searchsupport lines that are a few percent inward from each of P3 and P4,based on the center reference positions P1 to P4 of the center blockarea 16, as shown in FIG. 22. Thus the four edge detection points TUVWare detected.

The peripheral reference position detection section 156 detects theposition Y=yQ that acts as reference for the search in the lateraldirection on the upper side, based on whichever is the smaller value ofyT that is the Y-coordinate of the point T and yU that is theY-coordinate of the point U, together with y1 that is the Y-coordinateof the point P1. The edge detection section 184 detects the positionY=yR that acts as reference for the search in the lateral direction onthe lower side, based on whichever is the smaller value of yV that isthe Y-coordinate of the point V and yW that is the Y-coordinate of thepoint W, together with y2 that is the Y-coordinate of the point P2.

The peripheral reference position detection section 156 identifies thefour points P5 to P8 by searching outward on the lines Y=yQ and Y=yR inthe difference image from each of the intersections between the fourstraight lines X=xt, X=xU, Y=yQ, and Y=yR, and detecting pixels that areoutput. The peripheral reference position detection section 156identifies the remaining four points P9 to P12 by a similar method.

The projector 20 can identify the center reference positions of thecenter block area 16 and the peripheral reference positions of theperipheral block areas 17-1 to 17-4 by such a method, to identify thepositions of the four corners of the projection area.

In particular, such a method enables the projection area informationgeneration section 158 to obtain an approximated straight line in astate in which the three points for deriving that approximated straightline are further apart. This enables the projector 20 to detect theposition of the projection area to a higher precision.

Note that the numbers of the center reference positions and theperipheral reference positions can be set as required and are notlimited as in the above-described embodiments.

The patterns of the first calibration image 13 and the secondcalibration image 14 are also not limited to those shown in FIGS. 6A and6B, but it is preferable that the center block area 16 is formed in atleast the state in which the difference image is formed, and inparticular the center block area 16 and the peripheral block areas 17-1to 17-4 are formed in the state in which the difference image is formed.For example, the configuration could be such that the first calibrationimage 13 comprises the center block area 16 and the second calibrationimage 14 comprises the peripheral block areas 17-1 to 17-4.

In addition, the shapes of the calibration images, the center block area16, and the peripheral block areas 17-1 to 17-4 are not limited to beingrectangular; they could equally well be any shape other thanrectangular, such as circular. Of course the overall shapes of thecalibration images and the shape of the center block area 16 are notlimited to being similar; the shapes thereof could be such as to exhibita correspondence therebetween. In addition, the number of peripheralblock areas 17-1 to 17-4 is arbitrary.

The present invention is also effective in cases in which an image isprojected onto a projection target such as a blackboard or whiteboard,other than the screen 10.

The embodiments described above related to examples in which an imageprocessing system is mounted in the projector 20, but the imageprocessing system could equally well be mounted in an image displaydevice other than the projector 20, such as a cathode ray tube (CRT). Aprojector such as a digital micromirror device (DMD) could also be usedas the projector 20, other than a liquid-crystal projector. Note thatDMD is a trademark registered to Texas Instruments Inc. of the USA.

The functions of the above-described projector 20 could be implementedby the projector alone, by way of example, or they could be implementedby distributing them between a plurality of processing devices (such asbetween the projector and a PC).

In the above-described embodiments, the configuration was such that thesensor 60 was mounted within the projector 20, but the configurationcould also be such that the sensor 60 and the projector 20 are separateindependent devices.

Although only some embodiments of the present invention have beendescribed in detail above, those skilled in the art will readilyappreciate that many modifications are possible in the embodimentswithout materially departing from the novel teachings and advantages ofthis invention. Accordingly, all such modifications are intended to beincluded within scope of this invention.

1. An image processing system comprising: an image projection sectionwhich projects a first calibration image towards a projection target; asensing section which senses the projected first calibration image at alow resolution which is less than or equal to a predeterminedresolution, to generate first sensing information, and also senses thefirst calibration image at a high resolution which is greater than orequal to the low resolution, to generate third sensing information; anda projection-target area detection section which generatesprojection-target area information relating to a position of aprojection-target area corresponding to the projection target in asensing area of the sensing section, based on the first and thirdsensing information, wherein the projection-target area detectionsection comprises: an edge detection section which performs edgedetection to generate first edge detection information, based on thefirst sensing information, and also performs edge detection to generatethird edge detection information, based on the third sensinginformation; and a projection-target area information generation sectionwhich generates provisional detection information by provisionallydetecting the projection-target area based on the first edge detectioninformation, and also generates the projection-target area informationbased on the third edge detection information, wherein the edgedetection section generates the third edge detection information byperforming edge detection on a pixel group around a boundary line of theprovisionally detected projection-target area, based on the provisionaldetection information.
 2. The image processing system as defined inclaim 1, wherein the edge detection section detects edges at a pluralityof locations within a first sensed image based on the first sensinginformation, to generate the first edge detection information, andwherein the projection-target area information generation sectiongenerates the provisional detection information by setting a linearapproximated straight line or linear approximated curve, based onpositional information of the plurality of locations which is based onthe first edge detection information.
 3. The image processing system asdefined in claim 2, wherein the projection-target area detection sectioncomprises detection point evaluation section which evaluates a pluralityof edge detection points, and wherein the detection point evaluationsection determines whether or not each of the plurality of edgedetection points is distanced from the linear approximated straight lineor the linear approximated curve by at least a predetermined value, andcontrols the projection-target area information generation section insuch a manner that a detection point which is distanced by at least thepredetermined value is excluded and the linear approximated straightline or the linear approximated curve is reset.
 4. The image processingsystem as defined in claim 1, further comprising: a difference imagegeneration section which generates a difference image; and a centerreference position detection section which detects a plurality of centerreference positions of a predetermined center block area in the sensingarea, based on the difference image, wherein the projection-target areadetection section comprises search area determination section which setsan edge detection area by the edge detection means, wherein the imageprojection section projects a second calibration image, wherein thesensing section senses the projected second calibration image andgenerates second sensing information, wherein the difference imagegeneration section generates the difference image, based on the firstand second sensing information, wherein the search area determinationsection sets the edge detection area outside the center block area,wherein the first calibration image is a monochrome calibration image,and wherein the second calibration image comprises the center block areawhich is smaller than the second calibration image and which ispositioned around the center of the second calibration image.
 5. Theimage processing system as defined in claim 4, wherein the secondcalibration image is configured of the center block area, a peripheralblock area positioned on a periphery of the center block area, and abackground area that is an area other than the center block area and theperipheral block area, wherein each pixel in the center block area andthe peripheral block area has a different index value from each pixel inthe background area, the image processing system comprising: aperipheral reference position detection section which detects aplurality of peripheral reference positions of the peripheral block areain the sensing area, based on the center reference positions; and aprojection area information generation section which generatesprojection area information relating to a position of a projection areain the sensing area, based on the center reference positions and theperipheral reference positions.
 6. The image processing system asdefined in claim 5, wherein the projection area information generationsection generates the projection area information by setting a pluralityof approximated straight lines or approximated curves based on thecenter reference positions and the peripheral reference positions anddetermining the shape or arrangement of the center block area and theperipheral block area.
 7. The image processing system as defined inclaim 6, wherein the projection area and the center block area arerectangular areas, and wherein the projection area informationgeneration section determines positions of four corners of the centerblock area by deriving intersections of the plurality of approximatedstraight lines or intersections of the plurality of approximated curves,and generates the projection area information indicating positions offour corners of the projection area, based on the positions of the fourcorners of the center block area.
 8. The image processing system asdefined in claim 5, further comprising: a projection-target areaboundary point detection section which detects a plurality of boundarypoints of the projection-target area, based on the first sensinginformation and the center reference positions, wherein the peripheralreference position detection section detects the peripheral referencepositions, based on the boundary points.
 9. The image processing systemas defined in claim 5, further comprising: an image distortioncorrection section which determines whether there is distortion in animage projected by the image projection section, based on theprojection-target area information and the projection area information,and correcting an image signal to correct the distortion, wherein theimage projection section projects an image based on the image signalwhich has been corrected by the image distortion correction section. 10.A projector having the image processing system as defined in claim 1.11. A computer-readable storage medium having a computer-executableprogram embedded thereon, the program including computer executableinstructions, when executed by a computer, causing the computer to: animage projection control section which causes an image projectionsection to project a first calibration image towards a projectiontarget; a sensing control section which causes a sensing section tosense the projected first calibration image at a low resolution which isless than or equal to a predetermined resolution, to generate firstsensing information, and also for sensing the first calibration image ata high resolution which is greater than or equal to the low resolution,to generate third sensing information; and a projection-target areadetection section which generates projection-target area informationrelating to a position of a projection-target area corresponding to theprojection target in a sensing area of the sensing section, based on thefirst and third sensing information, wherein the projection-target areadetection section comprises: an edge detection section which performsedge detection to generate first edge detection information, based onthe first sensing information, and also performs edge detection togenerate third edge detection information, based on the third sensinginformation; and a projection-target area information generation sectionwhich generates provisional detection information by provisionallydetecting the projection-target area based on the first edge detectioninformation, and also generates the projection-target area informationbased on the third edge detection information, wherein the edgedetection section generates the third edge detection information byperforming edge detection on a pixel group around a boundary line of theprovisionally detected projection-target area, based on the provisionaldetection information.
 12. An image processing method comprising:projecting a first calibration image towards a projection target;sensing the projected first calibration image at a low resolution whichis less than or equal to a predetermined resolution by using a sensingsection, to generate first sensing information; sensing the firstcalibration image at a high resolution which is greater than or equal tothe low resolution by using the sensing section, to generate thirdsensing information; performing edge detection to generate first edgedetection information, based on the first sensing information;generating provisional detection information by provisionally detectinga projection-target area corresponding to the projection target in asensing area of the sensing section, based on the first edge detectioninformation; generating third edge detection information by performingedge detection on a pixel group around a boundary line of theprovisionally detected projection-target area, based on the provisionaldetection information; and detecting the projection-target area togenerate projection-target area information relating to a position ofthe projection-target area, based on the third edge detectioninformation.
 13. The image processing method as defined in claim 12,comprising: detecting edges at a plurality of locations within a firstsensed image based on the first sensing information, to generate thefirst edge detection information; and generating the provisionaldetection information by setting a linear approximated straight line orlinear approximated curve, based on positional information of theplurality of locations which is based on the first edge detectioninformation.
 14. The image processing method as defined in claim 13,comprising: determining whether or not each of a plurality of edgedetection points is distanced from the linear approximated straight lineor the linear approximated curve by at least a predetermined value; andresetting the linear approximated straight line or the linearapproximated curve in such a manner that a detection point which isdistanced by at least the predetermined value is excluded.
 15. The imageprocessing method as defined in claim 14, further comprising: projectinga second calibration image; sensing the projected second calibrationimage by using the sensing section, to generate second sensinginformation; generating a difference image, based on the first andsecond sensing information; detecting a plurality of center referencepositions of a predetermined center block area in the sensing area,based on the difference image; and setting an edge detection areaoutside the center block area, wherein the first calibration image is amonochrome calibration image, and wherein the second calibration imagecomprises the center block area which is smaller than the secondcalibration image and which is positioned around the center of thesecond calibration image.
 16. The image processing method as defined inclaim 15, wherein the second calibration image is configured of thecenter block area, a peripheral block area positioned on a periphery ofthe center block area, and a background area that is an area other thanthe center block area and the peripheral block area, wherein each pixelin the center block area and the peripheral block area has a differentindex value from each pixel in the background area, the image processingmethod further comprising: detecting a plurality of peripheral referencepositions of the peripheral block area in the sensing area, based on thecenter reference positions; and generating projection area informationrelating to a position of a projection area in the sensing area, basedon the center reference positions and the peripheral referencepositions.
 17. The image processing method as defined in claim 16,comprising: generating the projection area information by setting aplurality of approximated straight lines or approximated curves based onthe center reference positions and the peripheral reference positionsand determining the shape or arrangement of the center block area andthe peripheral block area.
 18. The image processing method as defined inclaim 17, wherein the projection area and the center block area arerectangular areas, the image processing method comprising: determiningpositions of four corners of the center block area by derivingintersections of the plurality of approximated straight lines orintersections of the plurality of approximated curves, and generatingthe projection area information indicating positions of four corners ofthe projection area, based on the positions of the four corners of thecenter block area.
 19. The image processing method as defined in claim16, comprising: detecting a plurality of boundary points of theprojection-target area, based on the first sensing information and thecenter reference positions; and detecting the peripheral referencepositions, based on the boundary points.
 20. The image processing methodas defined in claim 16, comprising: determining whether there isdistortion in an image to be projected, based on the projection-targetarea information and the projection area information, and correcting animage signal to correct the distortion.